The present invention is generally related to storage systems. More particularly, the invention is directed to a disk-based storage system which is optimized for storage and retrieval of sequential data.
Typical disk file systems such as the Microsoft NTFS, Solaris UFS and the Veritas VxFS are designed for the general-purpose storage of data. In particular, disk file systems are architected to handle the simultaneous reading and writing of a large number of both small and large files. To do this efficiently, file systems must implement intricate locking techniques, must be able to cope with random behavior and must be able to perform effectively under a variety of different conditions. This general-purpose design is adequate for some applications but can lead to significant overhead in complexity, performance and capacity.
In practice, storage intensive applications can be severely limited by general-purpose file systems. This is particularly true for applications that rely on mostly sequential data. Storing and retrieving data in a disk-based backup and data protection environment is an example of a specialized application that suffers from limitations in general-purpose file systems. First, general-purpose file systems typically only scale to less than one terabyte, whereas backup data sets can be significantly larger. Second, general-purpose file systems are fragile, in that they have difficulties in dealing with partial failures. Third, general-purpose file systems handle large streams of data inefficiently. Finally, general-purpose file systems, by their nature, cannot implement special purpose functionality because a feature that may be advantageous for one application may be disastrous for another.